Experimental demonstration of associative memory 
with memristive neural networks 
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Abstract — Synapses are essential elements for computation and 
information storage in both real and artificial neural systems. An 
artificial synapse needs to remember its past dynamical history, 
store a continuous set of states, and be "plastic" according to the 
pre-synaptic and post-synaptic neuronal activity. Here we show 
that all this can be accomplished by a memory-resistor {memristor 
for short). In particular, by using simple and inexpensive off- 
the-shelf components we have built a memristor emulator which 
realizes all required synaptic properties. Most importantly, we 
have demonstrated experimentally the formation of associative 
memory in a simple neural network consisting of three electronic 
neurons connected by two memristor-emulator synapses. This 
experimental demonstration opens up new possibilities in the 
understanding of neural processes using memory devices, an 
important step forward to reproduce complex learning, adaptive 
and spontaneous behavior with electronic neural networks. 

Index Terms — Memory, Resistance, Neural network hardware. 
Neural networks. 



I. Introduction 

WHEN someone mentions the name of a known person 
we immediately recall her face and possibly many 
other traits. This is because we possess the so-called asso- 
ciative memory - the ability to correlate different memories to 
the same fact or event This fundamental property is not 
just limited to humans but it is shared by many species in the 
animal kingdom. Arguably the most famous example of this 
are experiments conducted on dogs by Pavlov |2| whereby 
salivation of the dog's mouth is first set by the sight of food. 
Then, if the sight of food is accompanied by a sound (e.g., the 
tone of a bell) over a certain period of time, the dog learns to 
associate the sound to the food, and salivation can be triggered 
by the sound alone, without the intervention of vision. 

Since associative memory can be induced in animals and 
we, humans, use it extensively in our daily lives, the network 
of neurons in our brains must execute it very easily. It is 
then natural to think that such behavior can be reproduced 
in artificial neural networks as well - a first important step 
in obtaining artificial intelligence. The idea is indeed not 
novel and models of neural networks have been suggested 
over the years that could theoretically perform such function 
ID, iHl, fSl, 161. However, their experimental realization, 
especially in the electronic domain, has remained somewhat 



difficult. The reason is that an electronic circuit that simulates 
a neural network capable of associative memory needs two 
important components: neurons and synapses, namely connec- 
tions between neurons. Ideally, both components should be of 
nanoscale dimensions and consume/dissipate little energy so 
that a scale-up of such circuit to the number density of a typical 
human brain (consisting of about 10^° synapses/cm^) could be 
feasible. While one could envision an electronic version of the 
first component relatively easily, an electronic synapse is not 
so straightforward to make. The reason is that the latter needs 
to be flexible ("plastic") according to the type of signal it 
receives, its strength has to depend on the dynamical history 
of the system, and it needs to store a continuous set of values 
(analog element). 

In the past, several approaches with different levels of ab- 
straction were used in order to implement electronic analogues 
of synapses |7|. For instance, one of the first ideas involved 
the use of three-terminal electrochemical elements controlled 
by electroplating |8|. While some of these approaches do not 
involve synaptic plasticity at afl ID, GUI, |[TT1, |[l2l, die latter 
is generally implemented using a digital (or a combination of 
analog and digital) hardware fll. 041. fBI. fT6]|. ifTTl. ifTSl. 
|fT9l . The common feature of synaptic plasticity realizations 
is the involvement of many different circuit elements (such as 
transistors) and, therefore, occupation of a significant amount 
of space on a VLSI chip. Thus, the amount of electronic 
synapses in present VLSI implementations is much lower than 
the amount of synapses relevant to actual biological systems. 
Novel, radically different approaches to resolve this issue 
would be thus desirable. 

A recently demonstrated resistor with memory (memristor 
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Fig. 1. Artificial neural network for associative memory. Real neurons and 
their networks are very complex systems whose behavior is not yet fully 
understood. However, some simple brain functions can be elucidated studying 
significantly simplified structures. Here, we consider three neurons (A^i, N2 
and N3) coupled by two memristive synapses (Si and 52)- The output signal 
is determined by input signals and strengths of synaptic connections which 
can be modified when learning takes place. 
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for short (SOl, ET]) based on Ti02 thin films lH, (H offers a 
promising reahzation of a synapse whose size can be as small 
as 30x30x2 nm-^. Using Ti02 memristors, a fabrication of 
neuromorphic chips with a synapse density close to that of the 
human brain may become possible. Memristors belong to the 
larger class of memory-circuit elements (which includes also 
memcapacitors and meminductors) ||24|| . ||25l . namely circuit 
elements whose response depends on the whole dynamical 
history of the system. Memristors can be realized in many 
ways, ranging from oxide thin films 1221 . Il23l . Il26l to spin 
memristive systems [271. However, all these realizations are 
limited to the specific material or physical property respon- 
sible for memory, and as such they do not easily allow for 
tuning of the parameters necessary to implement the different 
functionalities of electronic neural networks. 

In the present paper, we describe a flexible platform al- 
lowing for simulation of different types of memristors, and 
experimentally show that a memristor could indeed function as 
a synapse. We have developed electronic versions of neurons 
and synapses whose behavior can be easily tuned to the func- 
tions found in biological neural cells. Of equal importance, 
the electronic neurons and synapses were fabricated using 
inexpensive off-the-shelf electronic components resulting in 
few dollars cost for each element, and therefore can be realized 
in any electronic laboratory. Clearly, we do not expect that 
with such elements one can scale up the resulting electronic 
neural networks to the actual brain density. However, due to 
their simplicity reasonably complex neural networks can be 
constructed from the two elemental blocks developed here and 
we thus expect several functionalities could be realized and 
studied. 

For the purpose of this paper we have built the neural 
network shown in Fig. [T] We have then shown that such 
circuit is capable of associative memory. In this network, 
two input neurons are connected with an output neuron by 
means of synapses. As an example of the functionality that 
this network can provide, we can think about the animal 
memory we have described above f2\ in which the first input 
neuron (presumably located in the visual cortex) activates 
under a specific visual event, such as "sight of food", and 
the second input neuron (presumably located in the auditory 
cortex) activates under an external auditory event, such as a 
particular "sound". Depending on previous training, each of 
these events can trigger "salivation" (firing of the third output 
neuron). If, at a certain moment of time, only the "sight of 
food" leads to "salivation," and subsequently the circuit is 
subjected to both input events, then, after a sufficient number 
of simultaneous input events the circuit starts associating the 
"sound" with the "sight of food", and eventually begins to 
"salivate" upon the activation of the "sound" only. This process 
of learning is a realization of the famous Hebbian rule stating, 
in a simplified form, that "neurons that fire together, wire 
together". 

II. Results and Discussion 

A. Electronic neuron 

Biological neurons deal with two types of electrical signals: 
receptor (or synaptic) potentials and action potentials. A stimu- 




Fig. 2. Electronic neuron, a. Main components of the electronic neuron 
proposed here are an analog-to-digital converter (ADC) and a microcontroller 
NC means not connected. If the voltage value Vin on the input terminal 
(I) exceeds a threshold voltage Vt (in our experiments Vt = 1-5V), the 
microcontroller connects the input pin to -2.5V and output pin (O) to 2.5V 
for 10ms, thus sending forward and backward pulses. After that, it waits for 
a certain amount of time St and everything repeats again. If Vin < Vt, 
then the microcontroller just continuously samples Vi„. The waiting time 
was selected as 5t = r — ■y ■ {Vi„ — Vt) + ^{v ~ 0-5), where t = 60ms, 
7 = SOmsA', A = 10ms and ?} is a random number between and 1. In 
our experimental realization, we used microcontroller dsPIC30F2011 from 
Microchip with internal 12bits ADC and a possibility of pin multiplexing, so 
that the only additional elements were two 10k resistors. Actual measurements 
were done using 5V voltage span, with further assignment of the middle 
level as zero. The value of resistors shown in a is lOkSl. b. Response of the 
electronic neuron on input voltages of different magnitude, the red dashed 
line is the threshold voltage and is only a guide to the eye. When Vi„ < Vt, 
no firing occurs. When Vin > Vt, the electronic neuron sends pulses, with 
the average pulse separation decreasing with increasing Vin- Vout is shifted 
for clarity. 



lus to a receptor causes receptor potentials whose amplitude is 
determined by the stimulus strength. When a receptor potential 
exceeding a threshold value reaches a neuron, the latter starts 
emitting action potential pulses, whose amplitude is constant 
but their frequency depends on the stimulus strength. The 
action potentials are mainly generated along axons in the 
forward direction. However, there is also a back-propagating 
part of the signal 1281 . |l29| which is now believed to be 
responsible for synaptic modifications (or learning) l29l . ||30l . 

We have implemented the above behaviour in our electronic 
scheme as shown in Fig. |2^ using an analog-to-digital con- 
verter and a microcontroller The input voltage is constantly 
monitored and once it exceeds a threshold voltage, both 
forward and backward pulses are generated whose amplitude 
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is constant (we set it here for convenience at 2.5V), but 
pulse separation varies according to the ampHtude of the input 
signal. In Fig. |2j5 we show the response of the electronic neu- 
ron when three resistors of different values are subsequently 
connected between the input of the neuron and 2.5V. When 
the resulting voltage (determined by the external resistor and 
internal resistor connected to the ground in Fig. |2^) is below 
the threshold voltage (t < Is in Fig.|2j)), no "firing" occurs (no 
change in the output voltage). When the input voltage exceeds 
the threshold (t > Is in Fig. Wp), the electronic neuron sends 
forward- and back-propagating pulses. The pulse separation 
decreases with increase of the input voltage amplitude as it is 
evident in Fig. 

B. Electronic synapse 

As electronic synapse we have built a memristor emulator, 
namely an electronic scheme which simulates the behaviour 
of any memory-resistor In fact, our memristor emulator can 
reproduce the behaviour of any voltage- or current-controlled 
memristive system. The latter is described by the following 
relations 



g{x,y,t)u{t) , 
X = f{x,u,t) , 



(1) 
(2) 



where y{t) and u{t) are input and output variables, such 
as voltage and current, g{x,u,t) is a generalized response 
(memresistance R, or memductance G), x is a n-dimensional 
vector describing the internal state of the device, and f{x, u, t) 
is a continuous n-dimensional vector function II2TII . ||25]| . 

As Fig.[3h illustrates schematically, our memristor emulator 
consists of the following units: a digital potentiometer, an 
analog-to-digital converter and a microcontroller. The A (or 
B) terminal and the Wiper of the digital potentiometer serve 
as the external connections of the memristor emulator The 
resistance of the digital potentiometer is determined by a code 
written into it by the microcontroller. The code is calculated by 
the microcontroller according to Eqs. ([T]i and (|2]). The analog- 
to-digital converter provides the value of voltage applied to 
the memristor emulator needed for the digital potentiometer 
code calculation. The applied voltage can be later converted 
to the current since the microcontroller knows the value of the 
digital potentiometer resistance. 

In our experiments, we implemented a threshold model 
of voltage-controlled memristor previously suggested in our 
earlier paper f31| and also discussed in Ref. |25|. In this 
model, the following equations (which are a particular case 
of Eqs. (1 1121) ) were used: 

G = (3) 

X = {j3VM + Q.h{a- P)[\Vm + Vt\-\Vm-Vt\]) 

xe{x~ Ri)e{R2-x), (4) 

where 9{-) is the step function, a and /3 characterize the 
rate of memristance change at \Vm\ < Vr and \Vm\ > Vr, 
respectively, Vm is a voltage drop on memristor, Vr is a 
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Fig. 3. Electronic synapse, a, Scliematic of tlie main units of the memristor 
emulator. The meiTuistor emulator consists of a digital potentiometer, ADC 
and microcontroller. The digital potentiometer unit represents an element 
whose resistance is defined by a digital code written in it. Two terminals 
of this unit (A and W) are the external connection terminals of the memristor 
emulator. The differential ADC converts the voltage between A and W 
terminals of the digital potentiometer into a digital value. The microcontroller 
reads the digital code from ADC and generates (and writes) a code for 
the digital potentiometer according to predefined functions g{x, u, t) and 
f{x, u, t) and Eqs. M\2\ . These operations are peii'ormed continuously. In 
our circuit, we used a 256 positions 10kf2 digital potentiometer AD5206 
from Analog Device and microcontroller dsPIC30F2011 from Microchip 
with internal 12bits ADC. b. Measurements of memristor emulator response 
when V(t) = Vq cos(27n.i;t) with Vq — 2V amplitude is applied to the 
circuit shown in the inset with i?o = lOkC The following parameters 
determining the meimistor emulator response (see Eqs. )3|4) ) were used: 
a = 13 = 146kQ/(V s), Vt = 4V, i?i = 675^, R2 = lOkQ. We noticed 
that the initial value of Rm (in the present case equal to lOkC) does not 
affect the long-time limit of the I-V curves. The signals were recorded using 
a custom data acquisition system. 



threshold voltage and Ri and R2 are limiting values of 
memristance. In Eq. (HJi, the 6'-functions symbolically show 
that the memristance can change only between Ri and i?2- In 
the actual software implementation, the value of x is monitored 
at each time step and in the situations when x < i?i or 
X > R2, it is set equal to Ri or R2, respectively. In this way, 
we avoid situations when x may overshoot the limiting values 
by some amount and thus not change any longer because of 
the step function in Eq. (|4]i. In simple words, the memristance 
changes between i?i and R2 with different rates a and (3 
below and above the threshold voltage. This activation-type 
model was inspired by recent experimental results on thin- 
film memristors |23J and as we discuss below it reproduces 
synapse plasticity. 
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Fig. 4. Development of associative memory, a. Using electronic neurons and electronic synapses (memristor emulators), we have built an electronic circuit 
corresponding to the neural network shown in Fig. [T] We have used the following parameters defining the operation of memristor emulators: Vt = 4V, 
a = 0, /3 = 15kf2/(V s). At the initial moment of time, the resistance of Si was selected equal to Ri = 675Q (lowest resistance state of memristor) and 
the resistance of 5*2 was selected equal to R2 = lOkSl (highest resistance state of memristor). Correspondingly, in the first probing phase, when Input 1 and 
Input 2 signals are applied without overlapping, the output signal develops only when Input 1 signal is applied. In the learning phase (see also b for more 
detailed picture). Input 1 and Input 2 signals are applied simultaneously. According to Hebbian rule, simultaneous firing of input neurons leads to development 
of a connection, which, in our case, is a transition of the second synapse S2 from high- to low-resistance state. This transition is clearly seen as a growth 
of certain pulses in the signal V3,i„ (voltage at the input of third neuron) in the time interval from 10.25s to lis in b. In the subsequent probing phase, we 
observe that firing of any input neuron results in the firing of output neuron, and thus an associative memory realization has been achieved. The curves in a 
and b were displaced for clarity. 



To test that our emulator does indeed behave as a memristor, 
we have used the circuit shown in the inset of Fig. [Sh, in 
which an ac voltage is applied to the memristor emulator, 
R, connected in series with a resistor Rq which was used 
to determine the current. The obtained current-voltage (1- 
V) curves, presented in Fig. [Sji, demonstrate typical features 
of memristive systems. For instance, all curves are pinched 
hysteresis loops passing through (0,0) demonstrating no energy 
storage property of memristive systems ET\ . ||24| . ||25l . More- 
over, the frequency dependence of the curve is also typical for 
memristive systems: the hysteresis shrinks at low frequencies, 
when the system has enough time to adjust its state to varying 
voltage, and at higher frequencies, when the characteristic 
timescale of system variables change is longer than the period 
of voltage oscillations. 

C. Associative memory 

Using the electronic neurons and electronic synapses de- 
scribed above, we have built an electronic scheme correspond- 
ing to the neural network depicted in Fig.[T] In this scheme, we 
directly connect the memristor terminals to the third neuron 
input. In such configuration, our network behaves essentially 
as a linear perceptron ll32l . Il33l . Il34l . although different 



connection schemes are possible. For example, putting a 
capacitor between the ground and input of the third neuron, 
we would obtain an integrate-and-fire model [35 1. In such a 
circuit (and also in perceptron networks with many synapses) 
it is important to ensure that the current from a synapse 
does not spread to its neighbors. This can be achieved by 
placing diodes between the right terminals of synapses in 
Fig. [T] and the third neuron's input. For our neural network 
containing only two synapses the effect of current spreading 
between synapses is not important. Moreover, we would like 
to highlight that our neural network is fully asynchronous, in 
distinction to a scheme suggested by Snider |30J| based on a 
global clock. This makes our approach free of synchronization 
issues when scaling up and closer to a bio-inspired circuit. 
An asynchronous memristor-based network was also discussed 
recently |36|. 

Fig. |4] demonstrates the associative memory development in 
the present network. Our experiment consists in application 
of stimulus signals to the first ("sight of food") and second 
("sound") neurons, and monitoring of the output signal on the 
third ("salivation") neuron. We start from a state when the 
first synaptic connection is strong (low resistance state of the 
first memristor) and second synaptic connection is weak (high 
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resistance state of the second memristor). 

In the first "probing phase" (t < 9s, Fig. |4| we apply 
separate non-overlapping stimulus signals to the "sight of 
food" and "sound" neurons. This results in the "salivation" 
neuron firing when a stimulus signal is applied to the "sight of 
food" neuron, but not firing when a stimulus signal is applied 
to the "sound" neuron. Electronically, it occurs because pulses 
generated by the "sight of food" neuron exceed the threshold 
voltage of the "salivation" neuron (due to a low resistance 
of the first memristor synapse) while the voltage on the 
"salivation" neuron input due to the "sound" neuron pulses 
is below the threshold voltage of the "salivation" neuron. 
In this phase there is no memristor state change since the 
first memristor is already in its limiting state (with minimal 
resistance allowed) and its resistance cannot decrease further, 
and voltage drop on the second memristor is below its voltage 
threshold. 

In the "learning phase" (9s < t <12s. Fig. |4]l, stimulus 
voltages are applied simultaneously to both input neurons, thus 
generating trains of pulses. The pulses from different neurons 
are uncorrected, but sometimes they do overlap, owing to a 
random component in the pulse separation (see Fig. |2] caption 
for details). During this phase, in some moments of time, 
back-propagating pulses from the "salivation" neuron (due 
to excitation from the "sight of food" neuron) overlap with 
forward propagating pulses from the "sound" neuron causing 
a high voltage across the second memristor synapse. As this 
voltage exceeds the memristor threshold, the second synapse 
state changes and it switches into a low resistance state. It 
is important to note that this change is possible when both 
stimuli are applied together (in other words they correlate). 
As a result, an association between input stimuli develops and 
the circuit "learns" to associate the "sight of food" signal to 
the "sound" signal. 

Our measurements during the second probing phase (t > 
12s, Fig. m clearly demonstrate the developed association. It 
is obvious that, in this phase, any type of stimulus - whether 
from the "sight of food" or from the "sound" neurons - results 
in the "salivation" neuron firing. 

Mention should be made about the memristor evolution 
function given by Eq. |4] In the present neural network the 
resistance of the second synapse can only decrease, since only 
negative or zero voltages can be applied to the memristor A 
process of synapse depression (corresponding to an increased 
synapse resistance in our electronic network) can be easily 
taken into account in different ways. The simplest way is to 
add a small positive constant to the first line of Eq. |4] (in 
the parentheses) for the conditioned memristor (S2 in Fig. [U. 
Then, even at zero value of Vj\/, the resistance of the memristor 
will slowly increase. Another way is to apply a small positive 
bias to the conditioned memristor (S2) in the circuit without 
any changes in Eq.|4](this would also require taking a non-zero 
a). 

III. Conclusion 

We have shown that the electronic (memristive) synapses 
and neurons we have built can represent important function- 
alities of their biological counterparts, and when combined 



together in networks - specifically the one represented in 
Fig. [U of this work - they give rise to an important function 
of the brain, namely associative memory. It is worth again 
mentioning that, although other memristors (e.g., those built 
from oxide thin films |37|) could replace the emulator we 
have built, the electronic neurons and synapses proposed 
here are electronic schemes that can be built from off-the- 
shelf inexpensive components. Together with their extreme 
flexibility in representing essentially any programmable set of 
operations, they are ideal to study much more complex neural 
networks (JE] that could adapt to incoming signals and "take 
decisions" based on correlations between different memories. 
This opens up a whole new set of possibilities in reproducing 
different types of learning and possibly even more complex 
neural processes. 
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